test: fix some wasi_testsuite cases
#6343
Conversation
When running `cargo run wasi_testsuite -- --nocapture`, some test cases generate files and folders that need to be removed for subsequent test runs to pass. These paths are all suffixed with `*.cleanup` so we walk the suite directory before and after execution to clean up these files.
This change fixes how directories are mapped from the host environment to the WebAssembly guest when the `wasi_testsuite` test is run. Previously, Wasmtime's `--dir` flag was used which expects the passed directory to be in the current working directory. In this case, though, the directories were buried down `tests/wasi-testsuite/wasi-common/...` so the correct flag is `--mapdir`. Switching to this flag allows a large number of the ignored tests to now pass.
| cmd.arg("--dir"); | ||
| cmd.arg(parent_dir.join(dir)); | ||
| cmd.arg("--mapdir"); | ||
| cmd.arg(format!("{}::{}", dir, parent_dir.join(dir).display())); |
There was a problem hiding this comment.
Instead of passing the raw directory in the spec test suite to each test, could the directory perhaps be copied to a temporary location and have the test run against that? That way the temporary directory clean up would removing any extra files and the clean_garbage filtering may not be necessary?
Alternatively though explicitly cleaning out files I think is ok, and if this is how the test suite runs upstream I think it's fine to mirror that as well.
There was a problem hiding this comment.
Well, right now I believe the other wasi-testsuite test runner (the Python one) expects to clean up any *.cleanup files so I propose we stick with that for now. I opened WebAssembly/wasi-testsuite#74 to actually write down what a test runner should do and I added a comment there about this. How about we change this logic once we decide what to do there?
After thinking about this some more, I took another look at fixing some issues with the test cases. By doing the following, many more test cases now pass:
--mapdirinstead of--dirSee the commit messages for more detail.